class: center, middle, inverse, title-slide .title[ # Dan’s Dating DilemmR ] .subtitle[ ## <i>Do you really Likert, is it is it wicked</i><a href="https://www.youtube.com/watch?v=khW5leL19SA"> 💃 </a> ] .author[ ### Dr Danielle Evans ] .date[ ### 8 Feb 2024 ] --- class: red <style type="text/css"> a { color: #009af5; font-weight: bold; } a.glossary { font-weight: bold; color: #8e7df1; cursor: help; position: relative; } .remark-inline-code { font-size: var(--code-inline-font-size); color: #4266ff; padding: 2px; } strong { font-weight: bold; color: black; } .remark-slide-number { color: black; opacity: 1; font-size: 0.9rem; } .hljs-github .hljs-string, .hljs-github .hljs-doctag { color: #333; } .hljs-github .hljs-literal { color: #333; } .hiddenFrame{ height:1px; width:1px; opacity: 0; } .red.remark-slide-content { background-color: #000000; font-size: 1rem; padding: 0; width: 100%; height: 100%; } <!-- .remark-container { --> <!-- background: #000000; --> <!-- margin: 0; --> <!-- overflow: hidden; --> <!-- } --> </style> <br> <img src="data:image/png;base64,#img/dd.jpg" width="75%" style="display: block; margin: auto;" /> --- ## Session Overview - **Designing Effective Questionnaires** + From ideas to items + Question & response types + Example items - **Designing Effective Interviews** + From ideas to interviews + Interview schedules + Successful probes & follow-ups + Piloting --- class: center, middle ## Part 1: Quantitative | Questionnaires --- ## Introduction - In psychology, we often want to measure things we cannot **directly observe** - Within **quantitative methods**, measurement refers to the way we record behaviours, feelings, attitudes, & characteristics in a **quantifiable sense** - We can measure variables like height & weight, but we can also measure **'unquantifiable' constructs** like 'happiness' or 'lecture-induced boredom' - We can make attempts to measure these feelings through **observation**... + I could count how many times you smile during a conversation, or how many times you yawn in the next hour - But often we rely on our participants' abilities to **self-report** their behaviours, feelings, attitudes, & characteristics instead <br> <div class="sweat" style="font-size:90%"> <p><b>But!</b> We need to make sure that we ask our participants the <b>right questions</b> so that they can <b>self-report accurately...</b> this applies to quant & qual methods! </p> </div> --- ## From Ideas to Items - We most often ask participants to self-report through **questionnaires**, BUT someone has to actually create those questionnaires at some point... - When designing any questionnaire, the first task is identifying and **defining** what topic(s) we want to be covered in the survey - The aim here is to break down **big ideas** into **smaller concepts** that can be measured with **specific items** in a questionnaire --- ## Dan's Dating DilemmR - As you might remember from our final skills lab in DS, I went on **1000 dates** last term just for you, in the name of ✨**science**✨ - In my observations, I've found that I am only interested in the guys that don't like me back **[everybody say 'aww']** - So it turns out, I like people who have a **fear of commitment**... --- class: center, middle <img src="data:image/png;base64,#index_files/figure-html/unnamed-chunk-4-1.png" width="80%" /> --- ## Meet The FOCkers - But, 1000 dates is a lot of effort right?! I need to focus more on the people that are **my type**, i.e., those with a high **F**ear **o**f **C**ommitment (**FOC**), and only go on dates with them to increase my dating success - *[A totally logical conclusion you can't tell me this isn't a great idea...]* - So, for my next study/dating adventuRe, I will screen all dates before meeting by giving them a **questionnaire** to complete - And I'll only select those who score highly on **FOC** to go on a date so I don't waste my time on guys that I'm not attracted to - So for our session today, we need to figure out how to measure someone's '**Fear of Commitment**' 🥰 - The next step is to brainstorm what **thoughts**, **feelings**, **behaviours**, or **characteristics** define our construct -- <div class="pc" style="font-size:90%"> <p><b>Task!</b> It's Kahoot time! Get your devices out ready! </p> </div> --- ## From Ideas to Items: Define #### So, what even is 'Fear of Commitment'? - Think about the: - Thoughts - Feelings - Behaviours - & characteristics that define 'Fear of Commitment' <br> <br> <br> <br> <div class="tu" style="font-size:90%"> <p><b>Top Tip!</b> Picture people at the two extremes of Fear of Commitment, what do they 'look' like? What do they think/feel? What behaviours do they display? </p> </div> ??? General Commitment Anxiety Fear of Loss of Independence Vulnerability Concerns Past Relationship Traumas Fear of Making the Wrong Choice Avoidance Behavior Communication Apprehension Future Planning Decision-Making Anxiety --- ## Question Types - So now we have an idea of what 'Fear of Commitment' is, we can design some questions to measure it... - For our next step, we need to decide what **type** of questions we're going to ask: - **Open Questions** + Empty text boxes - **Closed Questions** + Yes/no + Multiple choice + Ranking scales + Likert scales --- ## Question Types - So now we have an idea of what 'Fear of Commitment' is, we can design some questions to measure it... - For our next step, we need to decide what **type** of questions we're going to ask: - **Open Questions** + Empty text boxes - **Closed Questions** + Yes/no + Multiple choice + Ranking scales + **Likert scales** <br> .center[ ### *That's the way, aha, aha, I Likert* ] --- ## Likert Scales - Usually 5 or 7 points covering the two extremes with a neutral option in the middle e.g., + **Not very often** to **very often** + **Very untrue of me** to **very true of me** + **Not at all important** to **extremely important** + **Extremely unlikely** to **extremely likely** + **Strongly disagree** to **strongly agree** + **Never true** to **always true** <br> <div class="sweat" style="font-size:90%"> <p><b>Easy Error!</b> The responses must fit with the phrasing of the question or statement! For today, let's go with <b>strongly disagree</b> to <b>strongly agree</b> </p> </div> --- ## From Ideas to Items - Now we've decided on a **Likert scale** we can come up with our **items**! + **Items** in this context, refers to the different **statements** we want to include in our questionnaire - As a starting point, it can be helpful to look at how our construct has been measured previously, or how it's been measured in similar contexts - We can use those examples as inspiration for coming up with our own items (but don't steal!!) - Looking at existing measures can also help us identify what gaps there are in previous work that our questionnaire can fill <br><br> <div class="mb" style="font-size:90%"> <p><b>Keep in Mind!</b> We want our items to be <b>valid</b> and <b>reliable</b> measures of the specific construct we're interested in! 🎯 </p> </div> --- ## My Attempts - I had a go at coming up with my own items in the next few slides... - Using what you already know about **questionnaire design**, I want you to have a go at telling me what's right or wrong with them... - & think of some suggestions of your own! (I need all the help I can get to hand out this questionnaire before **Valentine's Day**! 😱) --- ## My Attempts .center[ #### 1. How would you describe your fear of commitment? <img src="data:image/png;base64,#img/LS.png" width="75%" /> ] <br><br><br><br><br><br><br><br><br><br><br> <div class="pc" style="font-size:90%"> <p><b>Task!</b> Good? Bad? What do you think? Post on Kahoot! </p> </div> --- ## My Attempts .center[ #### 2. I often discuss my long-term goals and expectations with my current partner. <img src="data:image/png;base64,#img/LS.png" width="75%" /> ] --- ## My Attempts .center[ #### 3. Discussing future plans with a partner makes me uncomfortable and I fear repeating the mistakes of past relationships. <img src="data:image/png;base64,#img/LS.png" width="75%" /> ] --- ## My Attempts .center[ #### 4. The prospect of protracted relational commitment elicits sentiments of encumbrance and constraint on my personal liberty. <img src="data:image/png;base64,#img/LS.png" width="75%" /> ] --- ## My Attempts .center[ #### 5. I don't feel no anxiety when contemplating not making short-term commitments. <img src="data:image/png;base64,#img/LS.png" width="75%" /> ] --- ## My Attempts .center[ #### *Select how much you agree with the following statement.* #### 6. The fear of choosing the wrong partner holds me back from committing. <img src="data:image/png;base64,#img/LS.png" width="75%" /> ] --- ## Some Final Q-Tips - Use **statements** not questions - Make sure your response options fit the wording of the item - Be specific, ask one thing per question - Avoid **double-negatives** & **double-barrelled** questions - Use **neutral language** - don't present one response as being more desirable - Avoid **jargon** - Consider your **sample**, are there any factors that might impact their understanding of your items? - **Pilot** your questionnaire to ensure it makes sense to others <div class="tu" style="font-size:90%"> <p><b>Top Tip!</b> Keep the image of the 'extremes' in mind when creating and refining your own questionnaire items! </p> </div> --- class: center, middle ## Part 2: Qualitative | Interviews --- ## Qualitative Methods - With **qualitative methods** we are usually trying to understand someone's personal experiences - We're more concerned with meaning and understanding **subjective experiences** - There are *many* different types of qualitative research, and a lot of them involve using **interviews** as a method of data collection - 3 different types of interview: + **unstructured** + **semi-structured** + **structured** - Could involve individuals OR groups <div class="mb" style="font-size:90%"> <p><b>Be Aware!</b> Both quantitative and qualitative methods are <b>subjective</b>, but we'll learn more about that through this module! 😁 </p> </div> --- ## From Ideas to Interviews - The process of coming up with questions for interviews is quite similar to questionnaires... - You firstly need to identify the research topic or area of interest - For unstructured interviews the research topic can be relatively general, for structured interviews it needs to be more focused - For more structured interviews, you would then create an **interview schedule** (or an **interview guide**) that contains the questions you plan to ask all of your participant(s) + For more **sensitive topics** you might also create a **distress protocol** based on the questions in your interview schedule (we'll talk more about ethics in interviewing next week!) --- ## From Ideas to Interviews - To help you come up with questions, think about what type of information you would like to obtain: **opinions**, **attitudes**, **knowledge**, **beliefs**, **behaviours**, **sensory**, **experiences**, **feelings**, **demographics**... - Create a bank of questions that cover each of these domains for your research topic, focusing on the **past**, the **present**, and the **future** - e.g. "Thinking back to when you were a child, how did you feel about transitioning to high school? (**feelings** in the **past**) - Aim for open questions that elicit **story-telling** - remember the goal is to understand your participants' experiences in depth - **Test them out** - think about how you might answer them yourself/go through the questions with a friend --- ## Let's give it a go! - So sticking with our theme, if we were interested in experiences of **overcoming Fear of Commitment in dating**, what interview questions could we ask? <br><br><br><br><br><br><br><br><br><br><br><br> <div class="tu" style="font-size:90%"> <p><b>Keep in mind!!</b> We want to know attitudes, knowledge, beliefs, behaviours, experiences, feelings, demographics etc, in the past, present, & future</p> </div> ??? How has your own past relationships influenced your attitude toward commitment? How do you communicate commitment fears with a potential partner, and what challenges have you faced in these conversations? Are there specific strategies or approaches you find effective in addressing commitment concerns openly? How does commitment fear influence your decision-making process in dating? Have you experienced personal growth or transformation in how you handle commitment fears over time? What strategies or self-reflection exercises have helped you navigate and overcome commitment fears? --- ## Swaying from the Schedule - Sometimes you might need to ask questions that are not in your **interview schedule** - Participants could offer up interesting points that you hadn't originally considered when creating your interview schedule, but now want to explore further - Or it could be that your participant isn't very forthcoming, or something they've said is inconsistent/unclear - In these cases, you can sway from the interview schedule to **probe** or **follow-up** on prior responses --- ## Successful Probes & Follow-ups - Avoid directly asking '**why?**' - it can come off as confrontational, instead, try using '**how**' - Pay attention to your participant and actively listen + Otherwise you might miss important opportunities to probe! - When you create your interview schedule, include some **answer-based paths** to take to help you prepare follow-ups and probes in advance + I.e., if the participant says **x**, then I'll ask them **y** - Be flexible when creating your interview schedule, sometimes things you don't anticipate might come up <br> <div class="tu" style="font-size:90%"> <p><b>Top Tip!</b> Be comfortable with silence and your participant will want to fill it! Give them space and time to think! </p> </div> --- ## Piloting - It's easier to clarify what you mean by a question during an interview because you're present and involved in the process (vs a questionnaire...) - But it's much better to avoid any misunderstanding from the start - participants may feel uncomfortable asking for clarity - You should **pilot** your interview questions (and questionnaires too) before recruiting participants for the main study - Some questions might turn out to be **easily misunderstood** or actually **irrelevant** to the research question - After piloting, you can revise any problematic questions to be more effective <br> <div class="mb" style="font-size:90%"> <p><b>Keep in Mind!</b> The process of designing a questionnaire or an interview schedule is very reflective & iterative! It requires testing & 'thinking time' 💭 </p> </div> --- ## Best Practice - Ask one question at a time, avoiding questions that elicit short answers - It's important to build **rapport** so start with simple questions to ease your participant into the interview + Choose questions that your participant won't find too difficult to answer, they're likely to feel a little nervous, & might need some time to warm up! - When moving between topics, tell your participant and give them the opportunity to add anything before you go onto the next question - Keep difficult or potentially **sensitive questions** towards the end of the interview (when you both feel more comfortable with the interview process) - Close the interview with an **opportunity for participants to add** any more information ("is there anything else you’d like to add before we end?") - Tailor the **language** and **pace** of the interview to your **sample** + Consider any factors (such as age, culture, language etc.,) that might affect how participants understand and respond to your questions --- ## Summary - We heavily rely on **self-report measures** in psychology, and so we need to design these measures to be **valid** and **reliable** + To get good data, we need to ask good questions!! - The process of creating questions (for surveys OR interviews) generally starts with **defining** our research topic or area of interest - We then want to design questions that look at the different **thoughts**, **feelings**, **behaviours**, **characteristics**, **opinions**, **attitudes**, & **experiences** related to our research topic - We should **pilot** any **questionnaires** or **interview schedules** before administering them to ensure there aren't any errors in understanding, and to make sure our questions are actually relevant to our topic, making any adjustments as necessary - We can then evaluate how 'good' our questions are with statistics, which we'll be covering in the next few weeks! Woohoo! 😁 --- ## *That's all - happy questioning!* .center[ <img src="data:image/png;base64,#img/giphy2.gif" width="60%" style="display: block; margin: auto;" /> [Give session feedback here!](https://forms.gle/ZyXAB7kZzUUyct9n6) ]